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1 TITLE OF THE INVENTION 

2 BRIGHTNESS ADJUSTING APPARATUS FOR STEREOSCOPIC CAMERA 
3 

4 BACKGROUND OF THE INVENTION 

5 1 . Field of the invention 

6 The present invention relates to an apparatus for 

7 automatically adjusting a balance of brightness of a stereoscopic 

8 camera. 

9 2. Discussion of the background art 

10 In recent years, a stereoscopic vehicle surrounding 

11 monitoring apparatus using a pair of left and right cameras 

12 (stereoscopic camera having solid image element like CCD) mounted 

13 on the vehicle has been interested by automobile engineers. To 

14 detect a distance to an object, first respective picture element 

15 or pixel blocks having coincidence of brightness are found in 

16 left and right images (hereinafter referred to as stereo matching) , 

17 then distance data are calculated according to the principle of 

18 triangulation from a relative deviation amount between both pixel 

19 blocks. Consequently, in order to calculate distance data with 

20 high reliability, it is necessary to balance the brightness 

21 between left and right cameras. 

22 With respect to this, Japanese Patent Applications 

23 Laid-open No. Toku-Kai-Hei 5-114099 and No. Toku-Kai-Hei 5- 

24 26554 disclose a technique in which variations of output image 

25 signals which are caused by the difference of the output 



1 



1 characteristic of stereoscopic cameras and the like, are 

2 corrected by referring to a lookup table. The lookup table is 

3 for changing gains and offset amounts of image signals and is 

4 stored in ROM of the system. Analogue image signals outputted 

5 from each camera are adjusted by the lookup table after being 

6 converted into digital signals by A/D converter. Thus, 

7 variations of image signals are corrected and the accuracy of 

8 the stereo matching is raised. 

9 However, according to the aforesaid prior art, the 

10 lookup table is established individually for a given stereoscopic 

11 camera in the manufacturing process of the camera such that output 

12 characteristics of the left and right cameras agree with each 

13 other. The output characteristics of the stereoscopic camera, 

14 however, gradually deviate from the initially set value due to 

15 use environment or aged deterioration. Even if the output 

16 characteristic is well-balanced at the initial stage, that 

17 balance will be lost gradually, that is, the precision of the 

18 stereo matching degrades due to aged deterioration. 
19 

20 SUMMARY OF THE INVENTION 

21 It is an object of the present invention to provide 

22 an apparatus for automatically adjusting a brightness balance 

23 of a stereoscopic camera to enhance the accuracy of monitoring 

24 around the vehicle. To achieve the object, the brightness 

25 adjusting apparatus comprises an adjusting means for adjusting 
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1 the brightness balance by varying a gain, a distance data 

2 calculating means for finding a pixel block having a brightness 

3 correlation with a pixel block of a reference image in a comparison 

4 image and for calculating a distance data based on a city block 

5 distance between both pixel blocks, a distance data assigning 

6 means for assigning the distance data to the pixel block of the 

7 reference image, a first evaluation window establishing means 

8 for establishing a first evaluation window composed of a plurality 

9 of pixel blocks in the reference image, a parallax calculating 

10 means for calculating a parallax based on the distance data, a 

11 second evaluation window establishing means for establishing a 

12 second evaluation window composed of a plurality of pixel blocks 

13 in a comparison image based on the parallax, a first evaluation 

14 value calculating means for calculating a first evaluation value 

15 representing a magnitude of an entire brightness of the first 

16 evaluation window, a second evaluation value calculating means 

17 for calculating a second evaluation value representing a 

18 magnitude of an entire brightness of the second evaluation window 

19 and a correcting means for correcting the gain so as to reduce 

20 the difference between the first evaluation value and the second 

21 evaluation value. 
22 

23 BRIEF DESCRIPTION OF THE DRAWINGS 

24 Fig. 1 is a block diagram showing a stereoscopic vehicle 

25 surrounding monitoring apparatus using an adjusting device 
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1 according to an embodiment of the present invention; 

2 Fig. 2 is a flow chart showing processes for adjusting 

3 gains according to a first embodiment of the present invention; 

4 Fig. 3 is a flow chart continued from Fig. 2; 

5 Fig. 4 is a diagram for explaining positions of a first 

6 and second evaluation windows according to a first embodiment; 

7 Fig. 5 is a diagram for explaining a searching range 

8 of a second evaluation window; 

9 Fig. 6 is a diagram for explaining an evaluation method 

10 of a horizontal brightness edge (variation of brightness) in a 

11 pixel block; 

12 Fig. 7 is a flow chart showing processes for adjusting 

13 gains according to a second embodiment of the present invention; 

14 and 

15 Fig. 8 is a diagram for explaining positions of a first 

16 and second evaluation windows according to a second embodiment. 
17 

18 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

19 Referring now to Fig. 1, a stereoscopic camera for 

20 imaging the surrounding scenery of a vehicle comprises a pair 

21 of CCD cameras 1, 2 disposed in the vicinity of a room mirror 

22 in the compartment. The CCD cameras 1, 2 are transversely mounted 

23 at a specified interval of distance. The camera 1 is referred 

24 to as a main-camera for obtaining reference images and is mounted 

25 on the right side when viewed from a driver. On the other hand, 
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1 the camera 2 is referred to as a sub-camera for obtaining 

2 comparison images and is mounted on the left side when viewed 

3 from the driver. Analogue images which are outputted from the 

4 respective cameras 1, 2 in a synchronous timing are adjusted in 

5 respective analogue interfaces 3 , 3 so as to coincide with the 

6 input range of the latter stage. A gain control amplifier (GCA) 

7 3a in the analogue interface 3 serves as adjusting a brightness 

8 balance of a pair of analogue image signals. The gains of 

9 respective amplifiers 3a, 3a are established to values according 

10 to gain indicating values GMAIN, GSUB which are outputted from 

11 a micro-computer 9. 

12 The pair of analogue images adjusted in the analogue 

13 interface 3 is converted into digital images having a specified 

14 number of graduations (for example, 256 graduations in the gray 

15 scale) by an A/D converter 4. The pair of digitalized images, 

16 that is stereo images, are subjected to processes such as a 

17 correction of brightness, a geometrical conversion of images and 

18 the like in a correction circuit 5. Generally, since there is 

19 greater or lesser degree of errors in the position of the 

20 stereoscopic cameras 1, 2, differences exist between left and 

21 right images. To remove these differences, affine transformation 

22 and the like is used to perform geometrical transformations such 

23 as rotation, parallel translation and the like. These processes 

24 ensure a coincidence of horizontal line which is an essential 

25 condition of the stereo matching between the left and right images . 
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1 Through these image processes, reference image data composed of 

2 512 pixels horizontally and 200 pixels vertically are generated 

3 from the output signals of the main camera 1. Further, comparison 

4 image data are generated from the output signals of the sub camera 

5 2. The comparison image data have the same vertical length as 

6 the reference image data and a larger horizontal length than that 

7 of the reference image data. For example, the comparison image 

8 data are composed of 640 pixels horizontally and 200 pixels 

9 vertically. These reference and comparison image data are stored 

10 in an image data memory 7, respectively. 

11 A stereo calculating circuit 6 calculates distance 

12 data based on the reference and comparison image data. Since one 

13 distance data is produced from one pixel block composed of 4 x 

14 4 pixels, 128 x 50 distance data are calculated per one reference 

15 image of a frame size. With respect to a given pixel block in 

16 a reference image, a corresponding pixel block in a comparison 

17 image is identified by searching an area having the same 

18 brightness and the same pattern as that given pixel block of the 

19 reference image (stereo matching) . The distance from the camera 

20 to an object projected in the stereo image is expressed as a 

21 parallax in the stereo image, namely a horizontal deviation amount 

22 between the reference and comparison images. Accordingly, the 

23 search is performed on the same horizontal line (epipolar line) 

24 as a j coordinate of the reference image. In the stereo calculating 

25 circuit 6, a correlation is evaluated between the object pixel 
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1 block and the searching pixel block while shifting a pixel one 

2 by one on the epipolar line. The correlation between the object 

3 pixel block and the searching pixel block can be evaluated by 

4 calculating a city block distance for example. Basically, a pixel 

5 block whose city block distance is minimum is a pixel block having 

6 the correlation. The parallax between the object pixel block and 

7 the pixel block having the correlation is outputted as a distance 

8 data. Since Japanese Patent Application Laid-open No. Toku- 

9 Kai-Hei 5-114009 discloses a hardware constitution for 

10 calculating the city block distance, detailed description is 

11 omitted in this document. Thus calculated distance data of one 

12 frame are stored in a distance data memory 8. 

13 The micro-computer 9 (recognition section 10) 

14 recognizes the road configuration (straight or curved road, 

15 curvature of road etc.), solid objects ahead of the vehicle 

16 (preceding vehicle, etc.) and the like. The recognition is 

17 performed based on the image data stored in the image data memory 

18 7 and the distance data stored in the distance data memory 8. 

19 Further, other information not shown in the drawings such as 

20 information from a vehicle speed sensor, a steering sensor, a 

21 navigation system and the like , is referenced when it is necessary . 

22 Specific approaches as to how to recognize the road configuration 

23 and solid objects are disclosed in Unexamined Japanese Patent 

24 Application No. Toku-Kai-Hei 5-265547. According to the result 

25 of the recognition, when it is judged that an alarm is desired, 
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1 a warning device 11 such as a monitoring apparatus, a speaker 

2 and the like operates to call a driver's attention. Further, by 

3 controlling a control apparatus 12 as needed, a vehicle control 

4 such as a shift-down of an automatic transmission, a slow-down 

5 of engine power, or a depression of brake pedal, is carried out. 

6 An operation according to a first embodiment will be 

7 described. A calculation section 13 of the micro-computer 9 

8 performs a feed-back adjustment of a gain of the gain control 

9 amplifier 3a according to the flowcharts shown in Figs. 2 and 

10 3. These flowcharts are carried out repeatedly in every cycle 

11 of a specified time interval. The calculation section 13 

12 calculates a main gain indicating value GMAIN for the main camera 

13 1 and a sub gain indicating value GSUB for the sub camera 2, these 

14 values are converted into analogue values by D/A converters 14, 

15 14 respectively and the converted analogue signals are inputted 

16 to the respective gain control amplifiers 3a, 3a. 

17 First, at a step 1, a brightness data Al of each pixel 

18 existing in a first evaluation window Wl established in the 

19 reference image is read. Fig. 4 is a diagram for explaining the 

20 establishment of the first evaluation window Wl and a second 

21 evaluation window W2 which will be described hereinafter. The 

22 first evaluation window Wl is constituted by 16 x 16 pixels and 

23 is fixed in a specified position ( (a, b) in coordinates) in the 

24 reference image expressed by i-j coordinates. Consequently, a 

25 brightness data Al having 256 pixels is read at this step. In 
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1 this embodiment, the position of the first evaluation window is 

2 established in a relatively central area of the reference image, 

3 aiming at an object located 20 to 30 meters ahead of the vehicle. 

4 Next, at a step 2, the second evaluation window W2 is 

5 found in the comparison image based on the distance data in the 

6 first evaluation window Wl . As described before, since one 

7 distance data is calculated per one pixel block of 4x4 pixels, 

8 the first evaluation window Wl contains 16 distance data di (i=l 

9 to 16) . Based on these distance data di , a histogram is prepared 

10 and a value observed most frequently (most frequently observed 

11 distance value) is found in the histogram. Letting the value be 

12 a parallax % between both windows Wl , W2 , the second evaluation 

13 window W2 having the same area (16 x 16 pixels) as the first 

14 evaluation window Wl is established at coordinates (a + %, b) in 

15 the comparison image as shown in Fig. 4. That is, the second 

16 evaluation window W2 is established being offset by the parallax 

17 in the horizontal direction from the position of the window Wl. 

18 As another method of calculating the parallax % , there is a method 

19 of using a mean value of a plurality of the distance data di in 

20 the evaluation window Wl . This method however sometimes has an 

21 inferior accuracy to the case of using the most frequently 

22 observed distance value. 

23 Thus calculated second evaluation window W2 has a 

24 correlation with the first evaluation window Wl with respect to 

25 the brightness characteristic. The distance data of a given pixel 
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1 block is a value indicating a correlation object of this pixel 

2 block (position of the correlation area in the comparison image) . 

3 That is, if the distance data is determined, the correlation 

4 object of a given pixel block can be determined. Accordingly, 

5 if the respective distance data in the first evaluation window 

6 Wl have an adequate accuracy, the parallax % calculated according 

7 to the aforegoing method is a highly reliable value showing a 

8 correlation object in the overall first evaluation Wl . 

9 On the other hand, it is possible to determine the 

10 position of the second evaluation window W2 without referring 

11 to the distance data di in the first evaluation window Wl. In 

12 this case, the degree of correlation is evaluated overall area 

13 of 16 x 16 pixels while the evaluation is performed for every 

14 pixel one by one, starting from the basic point, coordinates (a, 

15 b) of the fist evaluation window on the epipolar line (j=b) in 

16 the comparison image in the stereo matching direction (in this 

17 embodiment, rightwards) . When an area having a largest 

18 correlation is found, this area is established to be a second 

19 evaluation window W2 . However, this method has a defect that the 

20 calculation amount needed for searching the correlation object 

21 of the first evaluation window Wl substantially increases, 

22 compared to the aforegoing method in which the distance data is 

23 used. The use of the distance data di existing in the first 

24 evaluation window Wl makes it possible to determine the 

25 correlation object of the first evaluation window Wl with less 
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1 amount of calculation. 

2 The program goes from the step 2 to a step 3 , in which 

3 256 pieces of brightness data A2 existing in the second evaluation 

4 window W2 are read. In order to evaluate the magnitude of overall 

5 brightness of the evaluation windows Wl , W2 , a mean brightness 

6 AVE1 of the first evaluation window Wl and a mean brightness AVE 2 

7 are calculated respectively (step 4) . Here, the mean brightness 

8 AVE1 (or AVE2) is a mean value of the 256 brightness data Al (or 

9 A2) read in the step 1 (or step 3) . Further, thus calculated mean 

10 brightness AVE1 , AVE 2 are stored in RAM of the micro-computer 

11 9 (step 5) . 

12 When it is judged at a step 6 that 30 samples of the 

13 mean brightness AVE1, AVE 2 have been stored, the program goes 

14 to steps after a step 7, in which gain indicating values GMAIN, 

15 GSUB are subjected to adjusting processes. First, at the step 

16 7 correlation coefficients R for evaluating the crrelationship 

17 of the mean brightness AVE1, AVE 2 of respective stored 30 samples 

18 are calculated. When the respective samples are expressed in 

19 (AVEli, AVE2i i=l to 30) , the correlation coefficient R in the 

20 entire samples can be calculated according to the following 

21 formula. 

22 [Formula 1] 

23 

Z(AVEIi -AV!TXAVE2i -AVE2) / 3 O Z(AVEli -AVEl)(AVE2i -AVE2) 

24 V Z(AVEli -AVEf) 2 / 3 0 V Z(AVE2i -AVE2) 2 / 3 0 V Z(AVEli-AVEl) 2 V Z(AVE2i — AVE2) 

25 



11 



1 where AVE1 is a mean value of 30 samples of the mean brightness 

2 AVE1 and AVE2 is a mean value of 30 samples of the mean brightness 

3 AVE 2 . 

4 Thus calculated correlation coefficient R is always 

5 within the range of -l^R^l. Under the ideal condition, the 

6 correlation coefficient R is equal to 1 and respective sample 

7 points (AVEli, AVE2i) are on a straight line AVE2i = b *AVEli + 

8 AVE2i + a (this relation is referred to as a complete correlation) . 

9 In the real world, however, since the samples are affected by 

10 noises and the like, these sample points (AVEli, AVE2i) scatter 

11 and as a result the correlation coefficient R becomes small. 

12 Accordingly, it is possible to evaluate the reliability of the 

13 stored sample data AVEli, AVE2i by calculating the correlation 

14 coefficient (step 8) . In this embodiment, in case where the 

15 correlation coefficient R is equal to or smaller than 0.99, it 

16 is judged that the reliability of the stored data is low. In this 

17 case, the main gain indicating value GMAIN and the sub gain 

18 indicating value GSUB are not changed (step 15) so as not to make 

19 an improper adjustment of brightness balance. Furthermore, all 

20 of 30 samples stored in RAM of the micro-computer 9 are cleared 

21 (step 12) and the program goes to RETURN. On the other hand, in 

22 case where the correlation coefficient R is larger than 0.99, 

23 the stored data is judged to be reliable and the program goes 

24 to a step 9. 

25 At a step 9, the total amount of the difference of mean 
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1 brightness per each sample, SUM, a sum of the difference of the 

2 mean brightness per sample is calculated according to the 

3 following formula ( 1 ^ i^ 30 ) . 

4 [Formula 2] 

5 SUM = S (AVEli - AVE2i) 

6 The total amount of the difference of the mean 

7 brightness SUM is theoretically 0, if the brightness balance 

8 between the main camera 1 and the sub camera 2 is well-matched. 

9 However, in consideration of the stability of control, in case 

10 where the SUM is within a specified range (for example, -3500 

11 to +3500) , the present gain is judged to be in a proper condition. 

12 In this case, both the main gain indicating value GMAIN and the 

13 sub gain indicating value GSUB are not changed (steps 10, 13 and 

14 15) . 

15 On the other hand, in case where the SUM is smaller 

16 than a negative threshold value (-3500) , that is, in case where 

17 the comparison image outputted from the sub camera 2 is brighter 

18 than the reference image from the main camera 1, the program goes 

19 to a step 11 where 1 is added to the current sub gain indicating 

20 value GSUB and the main gain indicating value GMAIN is remained 

21 unchanged, that is, the current value GMAIN is used as it was. 

22 Since the added gain value makes the comparison image outputted 

23 from the sub camera 2 darker compared with the brightness before 

24 changing the gain, the unbalance of brightness between the cameras 

25 1, 2 is adjusted in a reducing direction. Thus, the difference 
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1 between the mean brightness AVE1 calculated in subsequent cycles 

2 of the first evaluation window Wl and the mean brightness AVE 2 

3 of the second evaluation window W2 becomes small. Then, the 

4 program goes to RETURN after at the step 12 the stored sample 

5 data are cleared. 

6 When the current sub gain indicating value GSUB is added 

7 by 1 , the sub gain indicating value GSUB sometimes goes beyond 

8 an allowable correction range (for example, -30 to +30) . In this 

9 case, the sub gain indicating value GSUB is unchanged. That is, 

10 instead of adding 1 to the sub gain indicating value GSUB, 1 is 

11 subtracted from the current main gain indicating value GMAIN. 

12 Since the reference image outputted from the main camera 1 

13 increases the brightness compared to the one before the change 

14 of gain, the brightness unbalance between the cameras 1, 2 is 

15 adjusted so as to be extinguished. Further, in case where both 

16 of the gain indicating values GMAIN, GSUB go beyond the allowable 

17 correction range, it is judged that the adjustment is impossible 

18 and neither values are not changed. 

19 On the other hand, in case where the SUM is larger 

20 than a positive threshold value (+3500) , that is, in case where 

21 the comparison image outputted from the sub camera 2 is darker 

22 than that of the main camera 1, the program goes to a step 14 

23 where I is reduced from the current sub gain indicating value 

24 GSUB and the main gain indicating value GMAIN is used as it is 
2 5 the current one. As a result, since the comparison image outputted 
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1 from the sub camera 2 becomes brighter than the one before changing 

2 the gain, the brightness unbalance is adjusted so as to be 

3 extinguished. Thus, the difference between the mean brightness 

4 AVE1 of the window Wl and the mean brightness AVE 2 of the window 

5 W2 becomes smaller. Then, the program goes to RETURN after the 

6 sample data stored is cleared at the step 12. 

7 When the current sub gain indicating value GSUB is 

8 reduced by 1, the sub gain indicating value GSUB sometimes goes 

9 beyond an allowable correction range. In this case, the sub gain 

10 indicating value GSUB is unchanged. That is, instead of reducing 

11 1 from the sub gain indicating value GSUB , 1 is added to the current 

12 main gain indicating value GMAIN. Since the reference image 

13 outputted from the main camera 1 is darker than the one before 

14 changing the gain, the brightness unbalance between the cameras 

15 1 , 2 is adjusted so as to be extinguished. 

16 Thus, since the feedback adjustment of the gain is 

17 performed in parallel with the monitoring control , the brightness 

18 balance of the stereo camera can be automatically adjusted. As 

19 a result of this, even if the initially set output characteristic 

20 of the stereo camera changes due to the aged deterioration or 

21 use environment, it is possible to adjust the balance of 

22 brightness of the stereo camera properly. The distance data 

23 calculated on the basis of thus obtained image signals can provide 

24 more accurate monitoring around the vehicle. 

25 Further, in this embodiment, the position of the second 
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1 evaluation window W2 which is the correlation object of the first 

2 evaluation window Wl is established based on the distance data 

3 existing in the first evaluation window Wl. Since the second 

4 evaluation window W2 is established in the position calculated 

5 from this distance data, a deviation of the brightness balance 

6 of the stereo camera can be detected accurately. As described 

7 before, the distance data calculated with respect to a given small 

8 region (pixel block) indicates a correlation object of the small 

9 region. Accordingly, the most frequently appearing distance 

10 value of the distance data existing in the first evaluation window 

11 Wl which is an assembly of small regions, represents an overall 

12 correlation object of the first evaluation window Wl . Thus, it 

13 is assured that both evaluation windows Wl, W2 have approximately 

14 the same brightness characteristics under the normal condition. 

15 In other words, an existence of a deviation of brightness between 

16 both evaluation windows Wl , W2 means that there is a brightness 

17 unbalance in the stereoscopic camera. 

18 Further, the method of establishing the second 

19 evaluation window W2 based on the distance data in the first 

20 evaluation window Wl can reduce the quantity of calculation 

21 substantially, compared to the method of finding the correlation 

22 object of the first evaluation window Wl by searching an entire 

23 comparison image. As a result, the micro-computer 9 does not need 

24 so large a capacity. Further, this method has an advantage of 

25 being able to adjust the brightness balance in real time in 
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1 parallel with the monitoring control around the vehicle. 

2 Further, in this embodiment, the reliability of the 

3 mean brightness AVE1, AVE 2 (sample data) is verified based on 

4 the correlation coefficient R. Only when it is judged that these 

5 sample data are highly reliable, the gain adjustment is executed. 

6 Accordingly, the gain adjustment can be performed properly 

7 without being affected by noises and the like. 

8 There are also the following variations of the 

9 aforementioned embodiment. 

10 (Variation 1) 

11 According to the embodiment described above, the 

12 parallax % is obtained from the distance data di in the first 

13 evaluation window Wl and then the position of the second 

14 evaluation window W2 is established as coordinates (a + % , b) 

15 based on the parallax %. That is, the position of the second 

16 evaluation window W2 is determined unconditionally from the 

17 calculated parallax % . On the other hand, according to a first 

18 variation, a searching range of the second evaluation window W2 

19 is established from the calculated parallax % and an area having 

20 a largest correlation in that range may be established as a second 

21 evaluation window W2 . Fig. 5 is a diagram for explaining the 

22 searching range of the second evaluation window W2 . Reference 

23 coordinates F (a + % , b) are determined based on the parallax 

24 % calculated from the distance data di of the first evaluation 

25 window. The searching range is established to be a range having 
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1 a specified width, extending on the epipolar line in the left and 

2 right direction respectively with reference to the reference 

3 coordinates F, that is, a range expressed in coordinates (a + 

4 % ± A , b) . In the stereo matching, there is a precondition that 

5 the correlation object of the reference image is located on the 

6 same horizontal line in the comparison image as the reference 

7 image. Accordingly, the correlation object of the first 

8 evaluation window Wl can be found by searching over this searching 

9 range. According to this method, the calculation quantity needed 

10 for searching in the correlation area increases compared to the 

11 first embodiment. However, this method has an advantage that even 

12 when the distance data existing in the first evaluation window 

13 Wl has an inadequate reliability, the correlation object of the 

14 first evaluation window Wl can be properly identified. 

15 (Variation 2) 

16 As described before, in the stereo matching, the 

17 distance data is calculated by finding the pixel block of the 

18 comparison image having a correlation with the brightness 

19 characteristic of the pixel block of the reference image. 

20 Accordingly, in case of the pixel block having no feature in the 

21 brightness characteristic, particularly in brightness edges, 

22 the stereo matching fails frequently and the reliability of the 

23 distance data of the pixel block is not so high. In view of this, 

24 it is desirable to calculate the parallax % using only the highly 

25 reliable data (that is , the distance data having brightness edges) 
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1 among the distance data di of the first evaluation window Wl. 

2 Fig. 6 is a view for explaining a method of evaluating brightness 

3 edges (variation of brightness) in the horizontal direction with 

4 respect to a pixel block. First, a variation (absolute value) 

5 of brightness Ap n (n = 1 to 16) of a pair of two horizontally 

6 adjacent pixels is calculated. With respect to the far left pixel 

7 line (P n , P 12 , P i3 , P i4 ) , a variation of brightness Ap is calculated 

8 from the far right line of a pixel block adjacent on the left. 

9 Next, the number of brightness variations exceeding a specified 

10 threshold value is counted from these 16 pieces of brightness 

11 variations. If the number of brightness variations exceeding the 

12 threshold value is equal to or smaller than 4, the pixel block 

13 has no specific feature in brightness and its distance data is 

14 judged to have a low reliability (invalid distance data) . On the 

15 other hand, if the number of brightness variations exceeding the 

16 threshold value is larger than 4, the distance data of the pixel 

17 block is judged to be highly reliable (valid distance data) . The 

18 parallax % is calculated based upon only the valid distance data 

19 among the distance data di in the first evaluation window Wl . 

20 The use of thus calculated parallax % provides an establishment 

21 of the second evaluation window W2 in a more appropriate position. 

22 Accordingly, it is possible to calculate the sample data AVE1 , 

23 AVE 2 having a higher accuracy. 

24 (Variation 3) 

25 In the first embodiment, the first evaluation window 



19 



1 Wl is fixed in a specified position. On the other hand, according 

2 to the variation 3, the position of the first evaluation window 

3 Wl may be varied. For example, an area having the largest number 

4 of the aforesaid valid distance data may be established to be 

5 a first evaluation window Wl . According to this method, since 

6 an area including the most reliable valid distance data is 

7 selected as a first evaluation window Wl , its correlation object 

8 can be precisely established. 

9 Fig. 7 is a flowchart showing a process for adjusting 

10 a gain according to a second embodiment of the present invention. 

11 In the flowchart, first at a step 21, brightness data 

12 Al of sub zones Rl , R2 , R3 (hereinafter, referred to as first 

13 zones) constituting the first evaluation window Wl in the 

14 reference image are read. Further, at a step 22, brightness data 

15 A2 of sub zones R4 , R5 , R6 (hereinafter, referred to as second 

16 zones) constituting the second evaluation window W2 in the 

17 comparison image are read. 

18 Fig. 8 is a diagram for explaining the establishment 

19 position of the first evaluation window Wl and the second 

20 evaluation window W2 . The first zones Rl , R2 and R3 positionally 

21 correspond to the second zones R4 , R5 and R6 , respectively. The 

22 positions of R4 , R5 and R6 of the second zones are established, 

23 in consideration of the stereo matching, being offset slightly 

24 from the positions Rl , R2 and R3 of the first zones in the direction 

25 of the stereo matching. The offset amount is established taking 
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1 a general tendency with respect to the distance to objects which 

2 would be generally projected in the first zones Rl , R2 and R3 

3 into consideration. 

4 When a vehicle monitors ahead of the vehicle during 

5 traveling, there is a tendency for the sky (infinite point) or 

6 solid objects in the relatively far distance (for example, 

7 buildings etc.) to be projected in the first zone Rl established 

8 on a relatively upper side of the reference image and in the second 

9 zone R4 corresponding to the first zone Rl in the comparison image . 

10 Accordingly, since parallaxes calculated in these zones Rl, R4 

11 tend to become relatively small, considering the tendency of the 

12 distance of solid objects and the like projected on the upper 

13 part of the image, the offset amount with respect to the second 

14 zones R4 is established to be smaller (or 0) beforehand. For 

15 example, as shown in Fig. 8, the second zone R4 is offset from 

16 the first zone Rl by the amount of 15 pixels in the stereo matching. 

17 Further, since generally, there is a tendency for 

18 vehicles traveling ahead of the self vehicle and the like to be 

19 projected on the first zone R2 established in the middle part 

20 of the reference image and the second zone R5 positionally 

21 corresponding to the first zone R2 , the parallax in the area tends 

22 to become medium. Consequently, taking the tendency of the scenery 

23 like this projected in the middle part of the image into 

24 consideration, the offset amount of the second zone R5 is 

25 established to be medium beforehand. According to the result of 
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1 experiments with respect to this, the offset amount is preferably 

2 established to be a parallax corresponding to the distance 30 

3 to 40 meters. For example, as shown in Fig. 8, the second zone 

4 R5 is offset from the first zone R2 by the amount of 25 pixels 

5 in the direction of the stereo matching. 

6 Further, since generally, there is a tendency for the 

7 ground surface such as roads and the like to be projected on the 

8 first zone R3 established in the lower part of the reference image 

9 and the second zone R6 positionally corresponding to the first 

10 zone R3 , the parallax in the area tends to become relatively large . 

11 Consequently, taking the tendency of the scenery like this 

12 projected in the lower part of the image into consideration, 

13 the offset amount with respect to the second zone R6 is established 

14 to be relatively large beforehand. For example, as shown in Fig. 

15 8, the second zone R6 is offset from the first zone R3 by the 

16 amount of 30 pixels in the direction of the stereo matching. 

17 Thus, the second zones R4, R5 and R6 are offset in the 

18 direction of the stereo matching in consideration of the general 

19 tendency of the distance to the objects projected in respective 

20 zones. As a result, since an identical scenery is projected 

21 respectively on a pair of zones (for example, Rl and R4) 

22 positionally corresponding of the reference image and the 

23 comparison image, both zones have almost the same brightness under 

24 normal imaging conditions. 

25 At a step 23, the mean brightness AVE1 of the first 
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1 evaluation window Wl and the mean brightness AVE 2 of the second 

2 evaluation window W2 are calculated respectively. To reduce the 

3 calculation quantity, the mean brightness AVE1 is calculated 

4 from the brightness data Al of every two horizontal lines in the 

5 first zone Rl , R2 and R3 . Further, similarly the mean brightness 

6 AVE 2 is calculated from the brightness data A2 of every two 

7 horizontal lines in the second zones R4 , R5 and R6 . The mean 

8 brightness AVE1 , AVE 2 calculated in a certain cycle are stored 

9 in the RAM of the micro-computer 9 (step 24) . 

10 The processes from the step 21 to" the step 24 are 

11 repeated in each cycle until 30 samples of the mean brightness 

12 data AVE1, AVE 2 are stored. When the 30 samples of the mean 

13 brightness data AVE1 , AVE2 are stored, the program goes from the 

14 step 25 of the cycle to the step 7 in the flowchart of Fig. 3. 

15 The processes after the step 7 are the same as those in the first 

16 embodiment and the description here is omitted. 

17 Also in this embodiment, similarly to the first 

18 embodiment, since the brightness balance of the stereoscopic 

19 camera can be automatically adjusted so as to be in a proper 

20 condition, the accuracy of the surroundings monitoring can be 

21 enhanced. 

22 Further, according to the second embodiment, 

23 differently from the first embodiment, the second evaluation 

24 window W2 is established without referring to the distance data. 
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1 Accordingly, the brightness balance can be effectively adjusted 

2 under the condition that the stereoscopic camera has a relatively 

3 large brightness deviation or positional deviation, that is, 

4 under the condition that this makes it impossible to calculate 

5 the highly reliable distance data. Such condition happens for 

6 example in a stage of the initial setting of at shipping of a 

7 stereoscopic camera or in an event of a readjustment thereof due 

8 to dead battery-backup or the like. 

9 While the presently preferred embodiments of the 

10 present invention have been shown and described, it is to be 

11 understood that these disclosures are for the purpose of 

12 illustration and that various changes and modifications may be 

13 made without departing from the scope of the invention as set 

14 forth in the appended claims . 
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1 WHAT IS CLAIMED IS: 

2 1. A brightness adjusting apparatus for adjusting a 

3 brightness balance of a pair of images outputted from a 

4 stereoscopic camera having a first camera imaging a reference 

5 image and a second camera imaging a comparison image, comprising: 

6 an adjusting means for adjusting said brightness 

7 balance by varying a gain; 

8 a distance data calculating means for finding a pixel 

9 block having a brightness correlation with a pixel block of said 

10 reference image in said comparison image and for calculating a 

11 distance data based on a city block distance between both pixel 

12 blocks; 

13 a distance data assigning means for assigning said 

14 distance data to said pixel block of said reference image; 

15 a first evaluation window establishing means for 

16 establishing a first evaluation window composed of a plurality 

17 of pixel blocks in said reference image; 

18 a parallax calculating means for calculating a 

19 parallax based on said distance data; 

20 a second evaluation window establishing means for 

21 establishing a second evaluation window composed of a plurality 

22 of pixel blocks in said comparison image based on said parallax; 

23 a first evaluation value calculating means for 

24 calculating a first evaluation value representing a magnitude 

25 of an entire brightness of said first evaluation window; 
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1 a second evaluation value calculating means for 

2 calculating a second evaluation value representing a magnitude 

3 of an entire brightness of said second evaluation window; and 

4 a correcting means for correcting said gain so as to 

5 reduce the difference between said first evaluation value and 

6 said second evaluation value. 
7 

8 2 . The apparatus according to claim 1 , wherein 

9 said second evaluation window is established in a 
10 horizontally offset position from said first evaluation window. 
11 

12 3. The apparatus according to claim 1, wherein 

13 said parallax is calculated based on a histogram of 

14 said distance data. 
15 

16 4. The apparatus according to claim 1, wherein 

17 said parallax is calculated based on a mean value of 

18 said distance data. 
19 

20 5. The apparatus according to claim 1, wherein 

21 said second evaluation window is established in a 

22 horizontally offset position by an amount of said parallax from 

23 said first evaluation window. 
24 

25 6. The apparatus according to claim 1, further 
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1 comprising; 

2 a correlation coefficient calculating means for 

3 calculating a correlation coefficient based on said first 

4 evaluation value and said second evaluation value. 
5 

6 7 . The apparatus according to claim 6 , wherein 

7 said first evaluation value and said second evaluation 

8 value are verified by said correlation coefficient. 
9 

10 8. The apparatus according to claim 1, wherein 

11 said second evaluation window is established by 

12 finding a pixel block having a largest brightness correlation 

13 with a pixel block of said first evaluation window in said 

14 comparison image within a specified range on the basis of a 

15 reference point established based on said parallax. 
16 

17 9. The apparatus according to claim- 1, wherein 

18 said parallax is calculated only based on said distance 

19 data of a pixel block having a larger variation of brightness 

20 than a threshold value. 
21 

22 10. The apparatus according to claim 1, wherein 

23 said first evaluation value and said second evaluation 

24 value are calculated from at least one pair of first and second 

25 zones prepared in said reference image and said comparison image, 
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1 respectively and said pair of zones are established being 

2 horizontally offset by an amount of pixels according to the 

3 position of said zones. 
4 

5 11. The apparatus according to claim 10, wherein 

6 said amount of pixels are established in consideration 

7 of a tendency of a distance to an solid object projected in said 

8 first zones. 
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1 ABSTRACT 

2 An apparatus for adjusting a brightness balance of a 

3 stereoscopic camera includes a gain control amplifier and a 

4 micro-computer. The gain control amplifier adjusts a brightness 

5 balance of a pair of images outputted from the stereoscopic camera 

6 by gain. The micro-computer calculates a first evaluation value 

7 representing a magnitude of an entire brightness of a first 

8 evaluation window established in a reference image outputted from 

9 the gain control amplifier and calculates a second evaluation 

10 value representing a magnitude of an entire brightness of a second 

11 evaluation window established in a comparison image outputted 

12 from an adjusting means and established in an area having a 

13 brightness correlation with the first evaluation window. Also, 

14 the micro-computer corrects a gain so as to reduce the difference 

15 between the first evaluation value and the second evaluation 

16 value. 
17 
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